Fast large scale Gaussian process regression using approximate matrix-vector products
نویسنده
چکیده
Gaussian processes allow the treatment of non-linear non-parametric regression problems in a Bayesian framework. However the computational cost of training such a model with N examples scales as O(N3). Iterative methods for the solution of linear systems can bring this cost down to O(N2), which is still prohibitive for large data sets. We consider the use of 2-exact matrix-vector product algorithms to reduce the computational complexity to O(N). Using the theory of inexact Krylov subspace methods we show how to choose 2 to guarantee the convergence of the iterative methods. We test our ideas using the improved fast Gauss transform. We demonstrate the speedup achieved on large data sets. For prediction of the mean the computational complexity is reduced from O(N) to O(1). Our experiments indicated that for low dimensional data (d ≤ 8) the proposed method gives substantial speedups.
منابع مشابه
Large-Scale Support Vector Machine Training using Low-Dimensional Approximation of the Kernel Matrix
Training Support Vector Machines (regression and/or classification) involves solving a simply constrained quadratic programming problem that scales linearly with training set size. Unfortunately this makes solving large-scale problems with millions of training samples difficult if not impossible from both computational and storage considerations. A standard approach is to exploit the sparsity o...
متن کاملGaussian Kullback-Leibler approximate inference
We investigate Gaussian Kullback-Leibler (G-KL) variational approximate inference techniques for Bayesian generalised linear models and various extensions. In particular we make the following novel contributions: sufficient conditions for which the G-KL objective is differentiable and convex are described; constrained parameterisations of Gaussian covariance that make G-KL methods fast and scal...
متن کاملApproximate Gaussian process inference for the drift function in stochastic differential equations
We introduce a nonparametric approach for estimating drift functions in systems of stochastic differential equations from sparse observations of the state vector. Using a Gaussian process prior over the drift as a function of the state vector, we develop an approximate EM algorithm to deal with the unobserved, latent dynamics between observations. The posterior over states is approximated by a ...
متن کاملSparse inverse kernel Gaussian Process regression
Regression problems on massive data sets are ubiquitous in many application domains including the Internet, earth and space sciences, and finances. Gaussian Process regression is a popular technique for modeling the input-output relations of a set of variables under the assumption that the weight vector has a Gaussian prior. However, it is challenging to apply Gaussian Process regression to lar...
متن کاملBeyond Classification – Large-scale Gaussian Process Inference and Uncertainty Prediction
Due to the massive (labeled) data available on the web, a tremendous interest in large-scale machine learning methods has emerged in the last years. Whereas, most of the work done in this new area of research focused on fast and efficient classification algorithms, we show in this paper how other aspects of learning can also be covered using massive datasets. The paper1 briefly presents techniq...
متن کامل